Pseudo supervision is regarded as the core idea in semi-supervised learning for semantic segmentation, and there is always a tradeoff between utilizing only the high-quality pseudo labels and leveraging all the pseudo labels. Addressing that, we propose a novel learning approach, called Conservative-Progressive Collaborative Learning (CPCL), among which two predictive networks are trained in parallel, and the pseudo supervision is implemented based on both the agreement and disagreement of the two predictions. One network seeks common ground via intersection supervision and is supervised by the high-quality labels to ensure a more reliable supervision, while the other network reserves differences via union supervision and is supervised by all the pseudo labels to keep exploring with curiosity. Thus, the collaboration of conservative evolution and progressive exploration can be achieved. To reduce the influences of the suspicious pseudo labels, the loss is dynamic re-weighted according to the prediction confidence. Extensive experiments demonstrate that CPCL achieves state-of-the-art performance for semi-supervised semantic segmentation.

知识蒸馏（KD）显示了其对象检测的有效性，在AI知识（教师检测器）和人类知识（人类专家）的监督下，它在该物体检测中训练紧凑的对象检测器。但是，现有研究一致地对待AI知识和人类知识，并在学习过程中采用统一的数据增强策略，这将导致对多尺度对象的学习有偏见，并且对教师探测器的学习不足，从而导致不满意的蒸馏性能。为了解决这些问题，我们提出了特定于样本的数据增强和对抗性功能增强。首先，为了减轻多尺度对象产生的影响，我们根据傅立叶角度的观察结果提出了自适应数据增强。其次，我们提出了一种基于对抗性示例的功能增强方法，以更好地模仿AI知识以弥补教师探测器的信息不足。此外，我们提出的方法是统一的，并且很容易扩展到其他KD方法。广泛的实验证明了我们的框架的有效性，并在一阶段和两阶段探测器中提高了最先进方法的性能，最多可以带来0.5 MAP的增长。

This paper studies the algorithmic stability and generalizability of decentralized stochastic gradient descent (D-SGD). We prove that the consensus model learned by D-SGD is O(m/N +1/m+λ 2 )-stable in expectation in the non-convex non-smooth setting, where N is the total sample size of the whole system, m is the worker number, and 1−λ is the spectral gap that measures the connectivity of the communication topology. These results then deliver an2 ) in-average generalization bound, which is nonvacuous even when λ is closed to 1, in contrast to vacuous as suggested by existing literature on the projected version of D-SGD. Our theory indicates that the generalizability of D-SGD has a positive correlation with the spectral gap, and can explain why consensus control in initial training phase can ensure better generalization. Experiments of VGG-11 and ResNet-18 on CIFAR-10, CIFAR-100 and Tiny-ImageNet justify our theory. To our best knowledge, this is the first work on the topology-aware generalization of vanilla D-SGD. Code is available at https://github.com/Raiden-Zhu/ Generalization-of-DSGD.

ProtoPNet and its follow-up variants (ProtoPNets) have attracted broad research interest for their intrinsic interpretability from prototypes and comparable accuracy to non-interpretable counterparts. However, it has been recently found that the interpretability of prototypes can be corrupted due to the semantic gap between similarity in latent space and that in input space. In this work, we make the first attempt to quantitatively evaluate the interpretability of prototype-based explanations, rather than solely qualitative evaluations by some visualization examples, which can be easily misled by cherry picks. To this end, we propose two evaluation metrics, termed consistency score and stability score, to evaluate the explanation consistency cross images and the explanation robustness against perturbations, both of which are essential for explanations taken into practice. Furthermore, we propose a shallow-deep feature alignment (SDFA) module and a score aggregation (SA) module to improve the interpretability of prototypes. We conduct systematical evaluation experiments and substantial discussions to uncover the interpretability of existing ProtoPNets. Experiments demonstrate that our method achieves significantly superior performance to the state-of-the-arts, under both the conventional qualitative evaluations and the proposed quantitative evaluations, in both accuracy and interpretability. Codes are available at https://github.com/hqhQAQ/EvalProtoPNet.

Open world object detection aims at detecting objects that are absent in the object classes of the training data as unknown objects without explicit supervision. Furthermore, the exact classes of the unknown objects must be identified without catastrophic forgetting of the previous known classes when the corresponding annotations of unknown objects are given incrementally. In this paper, we propose a two-stage training approach named Open World DETR for open world object detection based on Deformable DETR. In the first stage, we pre-train a model on the current annotated data to detect objects from the current known classes, and concurrently train an additional binary classifier to classify predictions into foreground or background classes. This helps the model to build an unbiased feature representations that can facilitate the detection of unknown classes in subsequent process. In the second stage, we fine-tune the class-specific components of the model with a multi-view self-labeling strategy and a consistency constraint. Furthermore, we alleviate catastrophic forgetting when the annotations of the unknown classes becomes available incrementally by using knowledge distillation and exemplar replay. Experimental results on PASCAL VOC and MS-COCO show that our proposed method outperforms other state-of-the-art open world object detection methods by a large margin.

Semantic segmentation based on sparse annotation has advanced in recent years. It labels only part of each object in the image, leaving the remainder unlabeled. Most of the existing approaches are time-consuming and often necessitate a multi-stage training strategy. In this work, we propose a simple yet effective sparse annotated semantic segmentation framework based on segformer, dubbed SASFormer, that achieves remarkable performance. Specifically, the framework first generates hierarchical patch attention maps, which are then multiplied by the network predictions to produce correlated regions separated by valid labels. Besides, we also introduce the affinity loss to ensure consistency between the features of correlation results and network predictions. Extensive experiments showcase that our proposed approach is superior to existing methods and achieves cutting-edge performance. The source code is available at \url{https://github.com/su-hui-zz/SASFormer}.

Reinforcement Learning (RL) is a popular machine learning paradigm where intelligent agents interact with the environment to fulfill a long-term goal. Driven by the resurgence of deep learning, Deep RL (DRL) has witnessed great success over a wide spectrum of complex control tasks. Despite the encouraging results achieved, the deep neural network-based backbone is widely deemed as a black box that impedes practitioners to trust and employ trained agents in realistic scenarios where high security and reliability are essential. To alleviate this issue, a large volume of literature devoted to shedding light on the inner workings of the intelligent agents has been proposed, by constructing intrinsic interpretability or post-hoc explainability. In this survey, we provide a comprehensive review of existing works on eXplainable RL (XRL) and introduce a new taxonomy where prior works are clearly categorized into model-explaining, reward-explaining, state-explaining, and task-explaining methods. We also review and highlight RL methods that conversely leverage human knowledge to promote learning efficiency and performance of agents while this kind of method is often ignored in XRL field. Some challenges and opportunities in XRL are discussed. This survey intends to provide a high-level summarization of XRL and to motivate future research on more effective XRL solutions. Corresponding open source codes are collected and categorized at https://github.com/Plankson/awesome-explainable-reinforcement-learning.

神经网络（NNS）和决策树（DTS）都是机器学习的流行模型，但具有相互排斥的优势和局限性。为了带来两个世界中的最好，提出了各种方法来明确或隐式地集成NN和DTS。在这项调查中，这些方法是在我们称为神经树（NTS）的学校中组织的。这项调查旨在对NTS进行全面审查，并尝试确定它们如何增强模型的解释性。我们首先提出了NTS的彻底分类学，该分类法表达了NNS和DTS的逐步整合和共同进化。之后，我们根据NTS的解释性和绩效分析，并建议解决其余挑战的可能解决方案。最后，这项调查以讨论有条件计算和向该领域的有希望的方向进行讨论结束。该调查中审查的论文列表及其相应的代码可在以下网址获得：https：//github.com/zju-vipa/awesome-neural-trees

原型零件网络（Protopnet）引起了广泛的关注，并增加了许多随访研究，因为它的自我解释特性可解释人工智能（XAI）。但是，当直接在视觉变压器（VIT）骨架上应用原始网络时，学到的原型存在“分心”问题：它们具有相对较高的可能性，即被背景激活，并且对前景的关注较少。建模长期依赖性的强大能力使得基于变压器的Protopnet难以专注于原型部分，从而严重损害了其固有的解释性。本文提出了原型零件变压器（ProtoPformer），以适当有效地应用基于原型的方法，并使用VIT进行可解释的图像识别。提出的方法介绍了根据VIT的建筑特征捕获和突出目标的代表性整体和部分特征的全局和局部原型。采用了全球原型，以提供对象的全球视图，以指导本地原型集中在前景上，同时消除背景的影响。之后，明确监督局部原型，以专注于它们各自的原型视觉部分，从而提高整体可解释性。广泛的实验表明，我们提出的全球和本地原型可以相互纠正并共同做出最终决策，这些决策分别忠实，透明地从整体和地方的角度缔合过程。此外，ProtoPformer始终取得优于基于原型的原型基线（SOTA）的卓越性能和可视化结果。我们的代码已在https://github.com/zju-vipa/protopformer上发布。

弱监督的对象本地化是一项具有挑战性的任务，旨在将对象定位具有粗糙注释（例如图像类别）。现有的深网方法主要基于类激活图，该图的重点是突出显示歧视性局部区域，同时忽略了整个对象。此外，基于变压器的技术不断地重点放在阻碍识别完整对象的能力的背景上。为了解决这些问题，我们提出了一种称为令牌改进变压器（TRT）的重新注意事项机制，该机制捕获了对象级语义，以很好地指导本地化。具体而言，TRT引入了一个名为令牌优先级评分模块（TPSM）的新型模块，以抑制背景噪声的效果，同时重点放在目标对象上。然后，我们将类激活图作为语义意识的输入合并，以将注意力图限制为目标对象。在两个基准测试上进行的广泛实验展示了我们提出的方法与现有方法的优势，该方法具有带有图像类别注释的现有方法。源代码可在\ url {https://github.com/su-hui-zz/reattentiontransformer}中获得。